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RESEARCH METHODS, IDEOLOCSf, AND VALUES 



We still do not know where the urge for truth comes 
from; for as yet we have heard only of the obligation 
imposed by society that it should axist; to be^ 
truthful means using the customary metaphors - in 
moral terras: the obligation to lie according to a 
fixed convention, to lie herd-like in a style obliga- 
tory for all .... 

Friedrich Nietzsche 

Lady, I do not make up things. That is lies. Lies 
are not true. But the truth could be made up if you 
know how. And that's the truth. 

Lily Tomlin 



Abstract 



There have been ongoing debates among educational researchers 
over the appropriateness of various research methods. This paper 
begins with the contention that these differences in 
methodological orientation are in large part reflections of the 
broader ideological and value convictions of the researchers 
themselves. A critical point to be made, however, is that 
methodological and ideological differences need not lead to 
unresolvable relativism in the search for meaning in research of 
all kinds. There are meaningful criteria for assessing a wide 
array of research conducted under divergent methodologies. In 
fact, common values and social purposes would se^ to underlie 
most, if not all, educational research in the United States. 
Ultimately, our first concern should be for the relevance of 
research findings. From this perspective, debate over preferred 
methodologies need not be "resolved" , since it is out of this 
dialectical, if untidy, process that values, me^ods, and 
application can mix to provide a closer connection between 
research and practice. 
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It is not a new insight to observe that quantitatively- 
oriented and qualitatively-oriented researchers have different 
focuses in defining, framing, and addressing research problems. 
These "paradigmatic"^ differences have led some methodologists to 
conclude that there may be a fundamental incompatibility of the 
criteria used by different research traditions to justify and 
evaluate research (Kuhn, 1962; Phillips, 1987; Winch, 1967). 

Others have examined these same issues and can find little 
empirical or logical basis to support such a conclusion (Howe, 
1988; Howe & Eisenhart, 1990; Garrison, 1986; Smith and 
Heshusius, 1986; Gage, 1989). 

The rhetoric characterizing this debate highlights the fact 
that although adherents of both viewpoints do often use different 
language, operate from differing assumptions, emphasize different 
perspectives on the phenomena under study, and employ different 
methods for collecting and processing information, they do 
generally aim to address the same or similar issues (Firestone, 
1987). Differences in method and vocabulary do not overshadow 
such common ground as the kinds of problems addressed, constructs 
of interest, the application of findings to practice and policy , 
and the fundamental values of beneficence, respect, justice, 
accuracy, and applicability (AERA, 1992; Miller & Fredericks, 

1991) . 

Sometimes these common interests and concerns are 
overshadowed by the apparent bitterness of the disagreement. 

It certainly makes no sense to argue that either quantitative or 
qualitative approaches to research are infallible or without 
limitations, and little would be gained by simply abandoning one 
set of methods for another. Why then, in spite of appeals to 
comity and methodological ecumenism, does there continue to be 
such fierce devotion to some methodological positions and enmity 
toward others? Can and should these differences be resolved? 

Foundations of 'Methodolotry"^ 

Clearly, much of the debate over methodological soundness or 
superiority is, in fact, rooted in ideological and rhetorical 
positions (Miller & Fredericks, 1991; Firestone, 1987). Research 
traditions provide methods or patterns for collecting and 
interpreting information that are logically consistent within the 
boundaries of a system of thought rationally derived from 
experience and reflection. Differences in the range of 
principles, attitudes, and beliefs that shape opinions of what 
constitute appropriate methodological moorings sustain the 
ongoing debates between adherents of traditionally quantitative 
approaches to research and adherents of traditionally qualitative 
approaches . 

From the contemporary vantage point these disagreements seem 
quite modern, or perhaps more precisely, post-modern; a product 
of post-world War II intellectual ferment and disaffection with 
scientific reductionism (e.g. - Kuhn, 1962; Winch, 1967; and 



Wittgenstein, 1968). However, from another perspective, the 
kinds of differences in the way researchers collect and process 
their information, in what they think is important about the 
world, lie deeply embedded in intellectual frameworks of very 
long pedigree.® 

For example, Kimball's (1989) analysis of Western 
intellectual traditions suggests that such debates may reasonably 
be seen as a continuation of the ancient struggle between the 
advocates of grammar and rhetoric (qualitative methods) and the 
advocates of mathematics and logic (quantitative methods) that 
can be traced to the Greeks of the fifth century B.C.E. From 
another perspective, Bordo (1986) has examined the profound 
impact of Descartes on Western thought as a basis of the 
historical and current intellectual dichotomy between those who 
emphasize connectedness (qualitative phenomenology) and those who 
emphasize detachment (quantitative objectivity) as ways of 
knowing. 

From such an historical perspective, the qualitative 
renaissance in educational research represents a resurgence of 
the rhetoricians' position as a counter-thesis to the twentieth 
century's dominant positivist or quasi— positivist* position. It 
is in the interest of all researchers to revitalize the 
discourses and disputes of the ancient rhetoricians and 
mathematicians. A more constructive approach to methodological 
differences than gainsaying the positions of those who operate 
from different assumptions seems to lie in considering these 
disputes as part of an ongoing intellectual dialectic that has 
shaped modern Western thought and forms the basis of all our 
attempts to systematically investigate and understand the world 
of experience. 



Bases for Evaluating Research 

Given this dialectical interpretation, methodological 
differences seem not to be implacable, but rather sides of the 
same coin, even complementary. Howe (1988) Laudan (1977), Miller 
and Fredericks (1991), Howe and Eisenhart (1990) and Tesch (1990) 
have amply illuminated the lack of any logical or technical basis 
from which to judge the superiority of one set of methods over 
another, or for rejection or preference for any given study 
solely on the basis of conformity to methodological preferences. 
Claims to the technical superiority of one methodological stance 
over all others are simply not logically tenable. If there is a 
general priority, it must be that research methods serve human 
inquiry and not the reverse. 

Does this spirit of detente mean "anyth' ng goes" - that is, 
it's all relative so let's just agree to disagree? I think not. 
Relativism has a kind of sickening circularity that does not seem 
to support some of the key values underlying research. Moreover, 
it seems qui e clear that some research is better than others . 
Further, relying solely on some form of consensus leaves the 
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quality of research to the mercy of group opinion, a practice 
that has come up short many times, if my reading of history is 
accurate . 

Let's consider the first set of these considerations, that 
relativism Is not the only alternative to methodological struggle 
and that there are bases for judging the quality of research, 
regardless of methodological framework. Both experience and 
reason, as elaborated by researchers and theorists from as 
disparate perspectives as Goodman (1978), Howe (1988), Eisner 
(1983; 1988), and Phillips (1987) reasonably lead us to 
ultimately reject the contention that because there are no 
methodological absolutes we must accept all conclusions or 
explanations as equally viable. 

What are appropriate and meaningful criteria from which we 
can make decisions about the quality and value of research?^ It 
is helpful to begin from the pragmatist position that the first 
purpose of research is problem solving (Laudan, 1977). From this 
perspective, it is rational to accept research and explanation 
that effectively solve problems of practice or theory, and 
irrational to do otherwise. Theoretical progress can be judged by 
the comprehensiveness of explanations and new predictions the 
research leads to; practical, or empirical, progress can be 
judged in terms of the empirical support found for phenomena, 
events, or relationships predicted by the research (Lakatos & 
Musgrave, 1972). This rationality is predicated on the 
effectiveness of the research in answering fundamental questions 
and accounting for observed phenomena. Thus, while prevailing 
doctrines, previous theories, and traditions (i.e. - "paradigms") 
shape an overall framework for a theory of rationality, 
individual research programs are judged specifically by their 
demonstrable, pragmatic success in addressing issues of 
importance to the research community, v/hich includes researchers, 
practitioners, and theorists. 

More specific criteria even more concretely linked to 
specific studies or research activities have been suggested by 
Howe (1988), Eisner (1983), Howe and Eisenhart (1990), and Miller 
and Fredericks (1991), among others. The first of these criteria 
revolves around the quality of the questions asked that form the 
basis of investigation. A centerpiece of good research is 
indisputably the asking of powerful and penetrating questions. 
Jackson (1990) has pointed out that pioneering works in 
sociology, psychology, and education have had sometimes serious 
flaws in terms of research methods and data analysis . Jackson 
thinks it is the quality of writing that made them important and 
influential; but a common thread is the quality and importance of 
the questions they asked and the quality of the thinking that 
preceded their search for answers. 

A necessary corollary to asserting the primacy of good 
questions for quality research is that the particular methods 
used to seek answers to these questions must be determined on the 



basis of how best to obtain information to provide meaningful 
answers. As Kaplan (1964), Maslow (1966), Shulman (1988), and 
Howe and Eisenhart (1990) have explained, research methods must 
be driven by the nature of the questions they are called upon to 
answer. Research methods are properly judged more by the degree 
to which they address the central purposes and contextual 
circumstances of the research being undertaken , and not by the 
degree to which they match the conventions or practices of some 
methodological orthodoxy (Howe & Eisenhart, 1990; Garrison, 

1986) . 

Following the pragmatists^ lead, then, the relative success 
of any given research effort is the degree to which initial 
questions are appropriately and thoroughly answered. There may be 
disagreements over what is an appropriate and thorough answer , 
but the best answers coherently account for all available 
observations, while lesser answers do not. This entails more 
than logical consistency or ”fit'' with observation, however. As 
Howe (1988) has noted, few theories have ever been abandoned 
because they were internally inconsistent, or because the 
language used to frame them could not be shaped to correspond 
with observation. A theory can be consistent but false, and 
multiple internally consistent theories can be developed to^ 
explain observed phenomena. Theories fail because as more is 
learned they become incapable of sustaining themselves as 
comprehensive, coherent, useful, and meaningful explanations of 
the phenomena they purport to address. 

A final observation on judging the merits of research qua 
research revolves around a fundamental need ior accountability. 
Thus, rigor and quality in research are framed in large measure 
by the degree to which researchers make processes of data 
collection, analysis, and interpretation as open as possible to 
public review and scrutiny (Constas, 1992; Garrison, 1986; Fish, 
1980; Howe & Eisenhart, 1990; Krathwohl , 1985; Phillips, 1987). 
While wide variations in method and data interpretation are 
unavoidable and perhaps even desirable, researchers are 
accountable to the audiences who will refer to their work for 
answers, insight, or understanding (Campbell, 1991; Eisner, 1983; 
Howe, 1988; and Miller & Fredericks, 1991). 

This seems to bring us to an apparent dilemma posed by 
constructivist theories of knowledge. Resort to a community of 
"organized skepticism" has long formed a cornerstone of 
scientific research, most clearly in the tradition of Popperian 
^conjectures and refutations' (Popper, 1964). Constructivists 
(Cuba, 1990; Phillips, 1995) generally hold that _ intersubjective 
agreement forms a basis for communal judgments with regard to 
what contributes meaningfully to knowledge and what does not. 
This process assists us in making meaning of the findings of 
individual research projects and of broader research programs. 
These meanings are never made in isolation, but mediated through 
the social and intellectual traditions that form the basis for 
interpretation of experience and ultimately, data in whatever 




>■'1 

i 



BEST COPY AVAIUBLE 



form they take (Eisner, 1990; Jacob, 1988; Phillips, 1995; 

Shapere, 1983; & MacIntyre, 1980). 

While this seems eminently reasonable, does resort to a 
universal critical community mean that research "truth" ^ is 
entirely a matter of persuasion and group agreement? Eisner 
(1983; 1990) has argued that the only difference between good 
reasons or poor ones for accepting an explanation of some 
phenomenon is consensus. In his words, "That might be all we can 
ever have, but we ought to recognize it for what it is" (1990, p. 
9). However, Eisner's conclusion omits the possibility that 
explanations and reasons for them might be tested in ways other 
than consensus. Consensus may be necessary but does not seem to 
be sufficient to warrant a claim to knowledge or an explanation 
of observed phenomena. 

As a simple example, several influential qualitative 
methodologists have repeatedly asserted that the Uncertainty 
Principle articulated earlier in this century by physicist Werner 
Heisenberg illustrates the universality of researcher effects on 
research participants and even environments (e.g. - Goetz & 
LeCompte, 1984; Lincoln & Cuba, 1985; Patton, 1980). However, as 
McKerrow and McKerrow (1991) have accurately pointed out, this 
p 0 i 7 spective is at best a distortion of Heisenberg's observation 
and at worst a gross misunderstanding of the principle. Though 
many researchers in education seem to believe that the Heisenberg 
principle refers to the act of observing as somehow changing what 
is observed, this is not what Heisenberg said. History is 
replete with examples of widely believed yet ultimately 
inaccurate ideas or of unscrupulous authorities working to 
"rewrite" history to their advantage. But neither broad 
agreement nor authoritarian propaganda can make fact of 
inaccuracy. The tenet that belief and accuracy are synonymous 
raises some troubling considerations, indeed. 

The discussion of group consensus brings into sharp relief 
the relationship between knowledge and belief. These constructs 
are products of interpretation, and interpretation is firmly 
grounded in the values that undergird all human inqiiry. 

Countless historical examples champion the necessity of the 
unfettered activity of a free and heterogeneous critical 
community. Consensus is an essential element of the dynamic of 
the dialectic of the critical community, however, consensus can 
never be accepted as the final settlement of an issue. 

Theoretical orientations, research approaches, and 
interpretation of data are not and have never been merely 
technical decisions, but are value-based decisions, the quality 
of which are rooted in attention to social needs, intellectual 
traditions, and the ongoing conversation among those engaged in 
the work of the critical community (Garrison, 1986). Even a 
cursory review of practice and theory in education reveals the 
core values common to all (or virtually all) research and 
decision making in education: 



First, there is a focus on individual growth and needs. This 
has taken shape through the methodological debate as recognition 
that people are purposive in their actions and engage in constant 
attempts to make meaning of their experiences and behave 
meaningfully in response to their experiences, corollary to this 
is further recognition that there are multiple factors and 
combinations of factors that influence human behavior and 
interpretation in and of the world and these complex interactions 
call for research efforts equal to the task (Salomon, 1991; 

Soltis, 1990b).® 

Second, there are professional commitments to beneficence, 
respect, and justice with regard to those at the center of any 
study (AERA, 1992; Soltis, 1990a). Differing paradigms, 
ideologies, or methodological biases aside®, these are 
fundamental values that assert the worth of each human life, its 
preservation, full development, and equality before the law that 
underscore the commitment to improving educational policy and 
practice. As Gage (1989) suggested, in the long run researchers 
will find that the "moral and rational foundations" of competing 
research paradigms are "virtually identical , dedicated to the 
same ideals of social justice and democracy and the goals of an 
education that would serve those ideals" (p. 8). 

Third, there is the long-standing commitment to be as 
faithful as possible to principles of accuracy. This includes 
attention to general notions of reliability and validity; to 
thorough and systematic information gathering; to thorough and 
clear explanation of processes of data collection, analysis, and 
the conclusions drawn from that analysis, why and how those 
conclusions were reached, and what these conclusions mean in the 
world of experience. 

Fourth, there is a commitment to educational research as an 
endeavor that can provide much needed insight into policy and 
practice in applied settings. Educational researchers find 
themselves in a field that is both complex and in urgent need of 
improvement. The commitment of the researcher to his or her art 
and science must be translated into research that addresses 
difficult and profound issues of policy, teaching, learning, 
finance, and social expectations. 

Conclusion 

Howe (1988), Garrison (1986), Smith and Heshusius (1986) and 
others have argued for an end to the "paradigm wars" on 
philosophical grounds. They have also illustrated reasons to be 
concerned if methodological dispute "prompts unreflectiveness and 
stifles progress" (Howe, 1988, p. 14), or if allowed to consume 
valuable time, energy, and other resources, contributes to 
neglect of concerns related to the "practical, everyday world of 
families, work, education, and government" (Gage, 1989, p. 8). 
Even researchers and theorists who have despaired of 
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general! zability (e.g. Cziko, 1989; Cronbach, 1975) have called 
foi‘ research that can support improvements in education policy 
and practice. The focus of research need not be on universal 
general! zability, but a more pragmatic concern with the 
usefulness and applicability of research findings (Donmoyer, 

1990). 

Research priorities in education must reflect broader 
social, political, and personal priorities. Students, teachers, 
administrators, parents, and all others with a stake in eduoa-ion 
at all levels are looking for answers and assistance. They 
frankly do not care much about paradigms, but will welcome the 
effort, talent, and time of the research community in addressing 
the complex issues and problems they face on a daily basis. 

Failure to embrace this role in providing ideas and answers to 
important questions risks increasing marginalization of 
researchers in realm of educational practice. Much ink has been 
spilled bemoaning the disconnection between r-^search and practice 
(Beyer & Trice, 1982; Keller, 1985; Lawler, et al, 1985). 
Acknowledging the primacy of our fundamental shared values over 
our methodological preferences will enhance, not hinder the 
research community as a partner in educational practice. 

The differences in perspectives and emphases of qualitative 
and quantitative approaches to research do not have to present 
roadblocks, but underscore opportunities for using multiple 
methods of complementary strengths (Creswell, 1994; Firestone, 
1987; Howe, 1988; Phillips, 1987; and Salomon, 1991). Multiple 
methods help us address different perspectives on issues, and 
can provide different kinds of knowledge in appropriate and 
useful ways that need not violate prevailing epistemological 
values. Even such quantitative luminaries as Cronbach (1975) and 
Campbell (1978) decades ago encouraged researchers to forge 
beyond the confines of positivism in its traditional sense. 

The dialectical interplay of methodological preferences and 
perspectives that goes on is a dynamic and beneficial 
characteristic of the current research landscape. The 
competition of ideas, theories, and lines of research are 
critical to the vitality of ongoing inquiry. Campbell (1991), 
for example, advocated the importance of maintaining "disputation 
about relative validity" when considering the merits of varying 
approaches to and philosophies of research (p. 587). On a 
grander scale, Thomas (1992), in a discussion of recent advances 
in medical science, strongly advocated the need for the ongoing 
free flow of information and even competition among the members 
of the research community. He did not advocate such competition 
in the sense that groups or individuals must somehow win or lose, 
but rather emphasized the synergistic quality inherent in the 
collaborative competition of "playing with delight against all 
odds in a huge endless game.... For science is only at its 
beginning, and almost everything important lies still ahead to be 
learned about and comprehended" (p. 182). How true for 
education, as well! 




lU 



Endnotes 



1. the very term 'paradigm' implies a much more profound 
difference in mental frameworks than the differences between most 
researchers in education. A genuinely different paradigm from 
mine is evident in the Aztec belief that human sacrifice was 
necessary to maintain the rhythms of the sun's rise and set. In 
this example, I can cognitively understand the statement I have 
just typed, but am apparently incapable of conceiving of the 
^^orld in such a way. That seems to be a more clear example of 
incoiamensurability, not mere disagreement on methods or what data 
are more valuable in answering important questions. The decision 
to drive a Ford instead of a Chevrolet is not a paradigmatic 
decision, but it is a choice based in values and preferred 
criteria. Furthermore, although incommensurability has been 
suggested as a fundamental barrier between denizens of different 
paradigms, even Kuhn (1962), to whom much of the 

incommensurability conversation is traced, has acknowledged that 
it is not necessarily the case that a person grounded in one 
'paradigm' cannot operate within another (Campbell, 1991). 

2 . Methodolotry is a term used to suggest the powerful 
devotion to certain methodological positions held by some 
researchers. When method is imperative above 9-11 other research 
considerations something would seem to be operating beyond 
logical selection of the best strategies for answering the 
important questions at hand. 

3. Bordo draws upon the work of such writers as Gilligan 
(1982), Ruddick (1984), and Chodorow (1978) in suggesting that 
there is a "natural foundation for knowledge, not in detachment 
and distance, but in closeness, connectedness, and empathy. They 
(Gilligan, Ruddick, and Chodorow) find the failure of connection 
(rather than the blurring of boundaries) as the principle cause 
of breakdown in understanding" (p. 455). Similarly, Kimball thus 
describes the modern heirs of the western rhetorical tradition, 
to whom he refers as hermeneuticists, "this entire movement 
constitutes a challenge to scientific understanding, normally 
conceived, and is closely linked to the desire to identify norms 
and values of human endeavor. While conceding the tremendous 
advances that the sciences have made in understanding natural 
phenomena, the new hermeneutics challenges the claims of 
scientific knowledge to control of a neutral, objective method 
insofar as it applies to the study of human activities and 
endeavors** (pp* 604~605). Kimball further notes that this 
hermeneutic movement wishes to substitute interpretive method, 
however tentative and uncertain, for the scientific method in the 
analysis of •social .nd human endeavor.... [and]... questions the 
relationship between theorizing and the social and political 
reality that theorists claim to describe, understand, or predict* 
( p. 605) . 

4. As Phillips (1987) has illustrated, even the meaning of 
the term 'positivism' as it seems to be used by different 
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researchers varies widely. In addition, it is not clear that 
very many researchers in education today really consider 
themselves logical positivists, but hold more or less behaviorist 
positions, perhaps. 

5. Salomon (1991) called for an end to paradigmatic conflict 
"because it diverts attention from another basic issue that 
transcends the guantitative~gualitative distinction. The issue 
stems from the realization, not a particularly new one, that 
classrooms (schools, families, therapies, cultures) are complex, 
often nested conglomerates of interdependent variables, events, 
perceptions, attitudes, expectations, and behaviors, and thus 
their study cannot be approached in tne same way that the study 
of single events and single variables can" (p. 11). 

6. Avoidance of "iatrogenic" conditions. If there is a 
clear ethical dimension to what we do as researchers, it must 
begin, as Soltis (1990) suggested, with that basic pledge of 
physicians; first, do no harm. This seems particularly important 
to those researchers who insist on an interventionist course. 
Those we seek to understand have their o\m cultures, 
understandings, and philo;. ^phies for living that we do not fully 
comprehend or we wouldn't be studying them in the first place. 
Researchers presume greatly who want to use their research to 
liberate, develop penetrative consciousness, or in some other way 
open the lives and minds of those they seek to better understand. 
There is a presumption of superiority in knowledge and 
understanding, a kind of benevolent and activist paternalism that 
aims to lift the veil of ignorance from the eyes of others. 
Researchers go into the field to learn from the people they want 
to understand, not to civilize, convert, transform, indoctrinate, 
or otherwise force them to be free (with apologies to Rousseau) . 
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